Optimizing time-shifts for reservoir computing using a rank-revealing QR algorithm. (arXiv:2211.17095v2 [cs.LG] UPDATED)
Reservoir computing, a recurrent neural network paradigm in which only the
output layer is trained, has demonstrated remarkable performance on tasks such
as prediction and control of nonlinear systems. Recently, it was demonstrated
that adding time-shifts to the signals generated by a reservoir can provide
large improvements in performance accuracy. In this work, we present a
technique to choose the optimal time shifts. Our technique maximizes the rank
of the reservoir matrix using a rank-revealing QR algorithm and is not task
dependent. Further, our technique does not require a model of the system, and
therefore is directly applicable to analog hardware reservoir computers. We
demonstrate our time-shift optimization technique on two types of reservoir
computer: one based on an opto-electronic oscillator and the traditional
recurrent network with a $tanh$ activation function. We find that our technique
provides improved accuracy over random time-shift selection in essentially all
cases.